• These slides show a proposed design of a Phase 2 trial with three experimental arms and no control

• The design was not used, so these slides were instead used as introduction to graphical hypothesis testing

• References to specific treatment regimens and disease setting have been removed

• Three arms: TrtA, TrtA + TrtB, TrtA + TrtC

  • Refer to these as mono, combo 1, and combo 2

• No control arm

• Planned \(N = 75\) patients per arm, total of \(225\) patients

  • may be reduced to \(N = 40\) per arm

• Futility analyis after 75 total patients have undergone sufficient follow-up time

• Meta analysis suggests a \(20\%\) objective response rate for patients on current standard of care

• ORR in each arm must have a confidence interval which lies above \(20\%\)

  • Equivalently, the ORR is tested with \(H_0: \pi_i = 20\%\) (exact binomial test, z-test, etc.)

A treatment must:

1. Improve expected ORR of 20% on standard of care

2. A combo must have significantly better ORR than monotherapy in addition to having better ORR than standard of care

First test ORR in each arm separately against 20% historical benchmark. Then test a combo against mono only if that combo better than standard of care.

• Tests of superiority relative to mono are gated behind tests against historical control

• Graphical testing procedure let you prioritize hypothesis tests using a straightforward, visual diagram while maintaining the FWER at \(\alpha\)

• You have \(M\) hypothesis tests, \(H_1, \dots, H_M\)

• Assign local significance levels, \(\alpha_1, \dots, \alpha_M\) to hypothesis test such that \(\sum_{j = 1}^{M} \alpha_j = \alpha\)

• When you reject a hypothesis test, \(H_j\), you can distribute its \(\alpha_j\) to other tests

  • You can reject these specific hypotheses at lower significance thresholds (Ex: \(0.025 \rightarrow 0.05\))

• Tests against historical control are given weight with \(\alpha / 3\)

• Tests of combo against mono are assigned initial weight of 0

• Promising arms can “give” \(\alpha\) to other arms.

• Graphical testing is performed in R version 3.6.0 using the gMCP, available through CRAN.
  • Direct link to the PDF instructions: https://cran.r-project.org/web/packages/gMCP/vignettes/gMCP.pdf
• Note: gMCP requires 64-bit Java. Set-up instructions here.
  • Alternatively, you can use the HPC environment

• R can open a GUI to manually construct graphical testing diagrams

library("gMCP")
graphGUI()

alpha  = 0.025
# weights for each hypothesis tests
weights <- alpha * c(1/3, 1/3, 1/3, 0,  0)
# Matrix of edges, shows how alpha can be shared
m <- rbind(c(0, 0.5, 0.5, 0, 0),
           c(0, 0, 0, 1, 0),
           c(0, 0, 0, 0, 1),
           c(1, 0, 0, 0, 0),
           c(1, 0, 0, 0, 0))
# Make a graph object
graph <- new("graphMCP", m = m, weights = weights)
# Run the graphical test 
test = gMCP(graph = graph, 
            pvalues = p_vals, #raw p-values from hypothesis test
            alpha = 0.025)

• Use simulations to see operating characteristics of our study

• See benefits of graphical tests:

  • Strict Type I error control with many tests
  • Increased power

• Objective response for arm \(j\) is drawn from \(\text{Binom}(n, \pi_j)\).

• Monotherapy success probability is varied from \(\pi_M = 0.20\) to \(\pi_M = 0.50\) in increments of 0.05.

• Combo 1 ORR success set \(\pi_{C1} = \pi_M + C\), where \(C = 0, 0.05, 0.10, \dots, 0.40\).

• \(\pi_{C1} \in [0.20, 0.90]\), depending on \(\pi_M\)

• For simplicity, combo 2 ORR success probability fixed at \(\pi_{C2} = \pi_M\) for all simulations.

• Assume a familywise Type I error rate of \(\alpha = 0.025\) for all 5 tests of ORR.

• The testing procedure controls the Type I error rate well below \(\alpha = 0.025\)

• ORR is tested against against a point null at 20% using an exact binomial test.

• The family-wise Type I Error Rate (\(\alpha\)) used in all simulations is \(0.025\)

• Tests in R are conducted with:

binom.test(
  x = num_OR, 
  n = 75,
  p = 0.20,
  alternative = "greater"
)

• Require one less response for each successful arms
  • \(\alpha\) is redistributed (“shared”) across arms

• Mono can be retested if the combo tests are rejected,
• The power of rejecting \(H_0: \pi_M \leq 20\) depends on \(\pi_C\)

• Power of rejecting \(H_0: \pi_{C_1} \leq \pi_{M}\).

• We may be concerned about isolation of effect if we don’t have a significant monotherapy

  • Ex: How do you know that TrtA isn’t driving efficacy alone?

• We want to show that monotherapy is efficacious and that the combos are more efficacious.

• Test this with the composite test: \(H_0: \pi_M \leq 20\%\) and \(H_0: \pi_{C1} \leq \pi_M\)

  • Test of efficacious monotherapy and combo superior to monotherapy